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A method is presented for the visual analysis of 
object? by computer. It is part leu la r 1 y we] I suited 
for opaque objects with smoothly curved surfaces. The 
method extracts information about the object's surface 
properties, "nelLd : nc measure? ■:■.•' ' (.-.. ■_, p l 1 ■-■ u -.i - " t y , 
texture* and 1 regularity. It also airk in determiri irigi 
the Object J s shape. 

The application of this method to 3 simple recog- 
nition task -- the recognition of Fruit -- is discussed, 
The results on a more complex smoothly curved object, a 
human face* are also considered. 



*This report reproduces a thesis- of the same title submitted 
to the Department of Electrical Engineering,, Massachusetts 
Institute of Technology, In partial Fulfillment of the re- 
quirements for the degree of Doclor of Philosophy, June 1570. 
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Chapter I The Problem 



Consider the problem of propi-amm 1 n-g a computer to 
recognize objects with smoothly curved surfaces, such as 
the object in the phoToaraph of figure- l_l* l^anes such 
as theje can be digit fled by an tma ca-d 1 ssec tor Camera^ 
so t!iet the picture Ts represented by p- raster o-* 
Inters it res at closely spaced sample points, represented 
numerically In figure 1,2 + We will consider s method of 
process! rg such input with tne ultimate goal of 
recognizing the object in the tmag9« 

There are numerous mere or less edeeuate known 
techniques tor classifying an image once significant 
features have teen extracted from Ft J but the problem of 
extracting sucti features frore the Dasic optical data is 
less well uncerstood. The methods which will &e 
discussed he»*e are 1 "l ow-l eve I n t In that they manipulate 
actual picture points and try to extract salient 
features, rather than working with high-level 
descriptions and attempting to produce on Ident I f icat 1 on, 

It must be rec-ogn 1 zed l however. That the so- 
called high' and low-level aspects of vision cannot 
really t?C cleanly separated. There ts no foolproO^ 
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Fi&uce 1*1: A Simple Smoothly Curved Object 




Figure I. 2s Sampled Light Intensities from the Apple 

of figure 1.1 



The intensities in thia array have been scaled to be between 

ft and 99 
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com p I e t e I y local way to find features, as t h £ r# * i i ! 
always be ambiguities which can only be resolved through 
the Lisa of cenrext. for example, one: must know the light 
[ntensTty let (east roughly) in order to determine 
whether an object Is- white or black, as e white object fn 
very dim light can easily reflect less light than a- black 
object Tn sunl ight» A plum cannot be easily 
d i 5t Ifigu Tshed from an isolated grape, unless the siie Is 
krtown + Highlights OH a sndoth surface cannot be 

understood unless the form of the Illumination 's known. 

The content can of course oe determined pertly 
from the scene 1 1 se I f , For example, a real scene wfll 
generally contain surfaces with a w lee range of 
reffectlvltfes. Thlsestablishasa Moht Intensity 
frame of reference fn which the lighter objects will 
appear white and the darker ones black. One cannot tell 
the sl^e of a white sphere alone in 2 pJiotOc rap h, but if 
It Is shown next to a tennis b?M , Its sTie Is known by 
comparison, (It Is possible^ but unlikely, that the 
tennis ba l l Is actually a scaled-up model three feet In 
diameter. ThTs usually happens only on movte sets,) >p 
a similar manner, the MghlTght on B known object gives 
Information about the lighting which can he used to 
Interpret the highlights on other ofcjeets In the Image* 



I ■ 

£o far, the u so of context has been considered 
only on the level of object Identification. Actual I y f 
context fs even more necessary at the I ev I of finding 
visual parts of objects, such as edces, h irne-finaing 
program can be saved an enormous amount of work If it Is 
told approximately where to look* If a program thinks it 
Is seeing an apple. It can know that a good way to verify 
this hypothesis Fs to (oak on top for a stem, 

A program can only make use of these cues, 
however. If It con pass information resulting from a 
partial identification back to the low*- 1 eve I feature- 
finding routines. This sort of system shall be referred 
to as "vert leal", In the sen&e that control passes 
■frequently between high- anc low-level routines. The 
term ,f hor 1 ^onte J " refers to a system which works Fn 
stages, each of which produces a more abstract 
representation of the scer-e, Much of the previous work 
In vision has been of this sort. A typical sequence 
might be to remove noTse, enhance features, extract 
features, group them, and then Identify objects. Since 
no provision Is made \n a hor frontal system for passing 
Information back down this chain, the system cannot make 
use of context Information obtained f rom the image 
Itself, 
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The methods which will be presented here ere 
Intended to fit in Td a vertical system in 'no ways. 
First, they can be used To start oft a vertical system 
with Information pjood enough to get it ^&[ng> Second, 
they extract features which are useful for object 
Identification. These features will ba extracted in 
such a m^nn^r as to allow easy advantage to be derived 
from context I nforuiet ion. 

This work Is intended to be e step to^Brds making 
computers see* This goa I is Interesting for a number of 
reasons. Computers with vision would be useful for 
applications In automat ion t and would fee able to Interact 
better with huifflnt* Corrpjter vision may weir provide 
Instructive models for the undersl aid 1 ng of hunan vision. 
The problem Is also very interesting In Its own rlcht, as 
en aspect of the study of Artificial I nte I I Ifjence . 



Chapter 7. Previous Work 



techniques have been Invest (gated which could be 
applied to smoothly curved objects as a step towards 
recogn ft Ion* 

2,1 Shape from Shading 

It Is possible to f f nd a great deal about the 
shape of e smoothly curved object from a single monocular 
image, given a knowledge of | + s surface reflection 
properties and the positron and nature of the light 
sources* Horn [|Q] generates curves tying on the 

surface of the object by an Iterattve solution of a set 
of differential equations re lat (ng shape to the Fntenslty 
of Image points* Similar methods have been applied to 
the analysis of funar tonography from Lunar Qrbfter 
pho^ograp hs \_ I 4 > b" . 

This method reqgfres a uniform object surface. 
Its reflectance must be a smooth -function of the angle 
the surface makes wTtti the Incident gnd ex It rays. Any 
marks on the surface will dr&rupt the solutions to the 
d [f f erentFa I ecua-'c-.s, although very small marks can be 
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Ignored by approprFate averaging tec hn ie/uas, 

Since his results depend so heav I Lv upon the 
Object surface properties, Horn C 1 Ol has. undertaken an 
investigation of hew typical surfaces bahave. To within 
a r ea so na bl e appro* rnat Ion, the rof I eetence of a real 
object car be considered to be a sFnplc linear 
superp ds ft i on of matte and specular components. Tie 
specular conporent refers to the light reflected such 
that the ancle Of Incidence equals the angle of 
reflection, as 1f the surface were mirrored* The matte 
component Is light which Is scattered by the surface, 
The simplest model 0+ a mattB surface Is a " I ambert la n" 
surface, which emits light uniformly in all directions, 
regardless of the incident angle. The physical effects 
which produce these two components are different, as 
shown by the fact that they frequently have different 
spectral properties* The specular component has the 
spectral distribution of the Incident I 3 [jht t while the 
matte component, which penetrates the surface more deeply 
before being scattered, is spectrally distributed as the 
product of the Incident light spectrum and the spectrum 
Of the object p foment. 



2.2 Detection of Optical Edges 

Much research has gone tii to the detect r on and 
tree Fog of contrast edges Fn an Inage, These edges cor 
he emphasised by d i i f eren 1 1 a t i on preer oeo ss Ing 
operations, such as the gradFenT or tap tat fan, 

2.2*1 Plane-surfaced Objects 



E d o, c defect Ton is particularly attractive f or 
plane surfaced objects* Since the edges are sTralght 
lines -I the intersection of two planes),, a determi net tofi 
of the position of the edges completely specifies the 
position 0* the plane surface vh \c h they enclose* and an 
ed£ c itself can be located 1 rt terms of just a f*w of Its 
poF nts, 

A progran by L. 0* Roberts recognises white plane 
surfaced objects on a dark background f I 5J. He cons Fdcrs 
objects which can be put together out of a set of given 
sub-shapes, such as rectangular p-ara I I e lop i ped a a nd 
wedges. The image Is first d 1 f f erent Fated * Lines are 
then found In the resulting picture by a mu 1 1 1 p le^step 
procedure, first fitting short lines to local areas, 
eflminatlng t[ny loops* then fitting longer and longer 
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lines to +fie shorter outs, and finally generating a 

I east -nta n- sauar e fine which is ta ker> to rapresant the 

or tcj F n3 I ed oe . 

The next phase Is recognftion of polygons in *ie 
line drawing! followed by the mntchtng Of sets- of 
polygons against the pes 5 I b I e mode I s. The matching is 
first -dona on a straight to pogra ph f c a I basis* The two- 
dlmensioraf projection of a brick, for instance, 
general ly con-tains three quadrilaterals with one corner 
point In Common, Kp> such point axtsts on a wedne + 
Assuming, then, that this point corresponds +0 the corner 
Of £• brick, tha program, can match the- ofher lines snd 
points in the quadr Hatora I s to what must then be the 
corresponding I fnes and points of the model. A leesT- 
tied r.-squar e error matrix procedure is then used to f J nc 
the best brick i tn i-space) which generates the given 
two-dimensional i \ ne drawing* If the ! east-mean-" a aua re 
error Ts small enough , the fit Is accepted as correct. 

Whan a set of Mncs are matched by a model, the 
model can then be projected back onto the line drawing, 
but now wFth all o-f tha hfdden lines present. The mode I 
Is now "removed" from the IFne drawing, which may entail 
the deletion of some lines, hut also may entail tha 
addition of soma others* The procedure Fs now iterated 



until all of the lines of the Incut -figure have b ee n 
accounted for. Thus objects are recognized as being 
compounded of a number of the basTc b u 1 I d f n n h!ccks, 

Roberts depends on a hi oh degree of precision of 
measurement of the position of the edges, since he uses 
perspective In an esse Tit fa I way. 'Jr. fortunately, his 
procedure Fs useless for objects lacking straight line 
edges* One particularly Interesting aspect of Roberts' 
work is Jits use of 3 powerful internal model of the 
potential object In the rmaoe. A similar approach m Eght 
be useful for scenes consisting of regular smoothly 
cu r ved objects such as spheres and cylinders, but it Is 
difficult to envision Successful results using rtvOr& 
amorphous forms* 

A program by R. W. Gosper visually locates white 
rectangular parallelepipeds on a black table* Cue to the 
high reflectance difference between the objects and the 
background, the outer ed-ges are very clearly defined* 
(The program also finds Interior edges of the object 
where the contrast between adjacent faces is -* I gi 
enough..) The edges are found by 3 n algorithm whTch scans 
1n a line p erpS nd Ecu lor to the edge, and moves this line 
along the ed ge f r on one end to the ot her, From the 
position of the edges En the \ m & ge , and the knowledge 



that "the object is r ec tannu lar and rests on a table vho sc 
position is known, the exact three-space position of the 
object can be calculated from its hexagonal gutffne, 

Griffith [a] has made a srudy of the edges o* 
geometrical objects, ant! has developed a theory of their 
optical detection in the presence of noise. His system 
Is designed to be easily integrated into a vertical 
vision system, and includes a high-level line proposer 
and vcr if Isr, 

Guzmen f9j "oerses M a stra f ght-l i ne drawing Into 
its component p ia ne-surfeeed 1 objects. His work Is 
notable here In that |t depends little upon accuracy Qf 
measur enent, and Is concerned with dissecting a complex 
scone Into individual objects pr Tor to determining their 
exact position or shape. In these respects It has goats 
sirc'ler to "li^iC of this *c r k with respect to smoothly 
curved objects. Guzman's techniques could In fact be 
extended to smoothly curved objects, ard used to 
complement the methods discussed In the next chapterj his 
methods &q not depend so much upon the edges being 
straight, as do Roberts and Gasper, 

Different recognition methods ca I I for different 
degrees of precision in tha final determination of line 
positron, Roberts requires hTgn precision, because of 
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his use of second-order per spec*" lv e effects* The use of 
stereo distance determination would also recirlre sutn 
h "g h ^rcci&iOrt. Gosper re qu Ires only ir-o d I urn precision. 
His gca I is to Actual ly pick up the blocH, which only 
requires locating 1+ to wFthin a centimeter or so. No 
perspective,, stereC; or other second -*ord«r effects are 
used, sg the celculeted position is not as senslf Fve to 
smg II errors In the I Fne pes It F on . T he programs of 
Gulman and Griffith rehire only low precision, exc*pt In 
a few parts which wake use of the para I I el 1 s«i of two 
I Fnes T 

2*2+? Curved Edges 



There has been much study of recognition of 
a I pharmmer Ic characters. Black characters on s whfte 
background provide high-contrast edges* and some 
cha racter-r ecogn f f Ion programs work by tracing around the 
character 1 s edge* There has teen rittle edge-or I ented 
research on images darFved from trirce-d Imens lona I 
objects* and the resists of the two-d lire ns lona I work has 
little relevance to this problem. 

It Is considerably Easier to fEntf s stratght edge 
than a CUfved one* since only two poFnts determine a 
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stra F ght line, and additional points csn then be ver i f : cd 
by very sensitive tests, If many tests art posTtlve 
along 3 straight Mne, the existence at the edge can then 
be asserted with a high statist tea! confidents, as by 
Griffith** programs* These Techniques can be used only 
over a short Interval for a curved edge* 

2,3 The "Rag tons" Approach 

Instead of looking for high-contrast edgesj sone 
pattern recognition methods fook tor homogeneous areas of 
low contrasi, AnaFysis then proceeds from the shape and 
Interelgt ions between these "regions'*. There are e 
number of techniques for charect er F z 1 ng the shape of a 
region* such as various moments C^H, or more complicated 
Shape descriptors Lil * Klrscn [ll] analyzes 
photomicrographs of cells L>y buildFng a tree structure of 
Image regions with various levels of homogeneity* His 
methods are the closest In the literature to those which 
ore developed Tn this thesis* 
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2 + 4 Textural Information 



T>e optical behavior of an object depends very 
much on the texture o* Its surface. The word h texfure n 
may refer to either markings or departures from a smooth 
surface, tut |n either case they must be smell compared 
with the size of the object In order to be considered 
texture. Texure analysis may be done by a w F d-5 variety 
of methods, such as Fourier analysis or cross- 
correlat Ion, Texture has bsen used to odvan>taae in a 
range of studies, In such areas as recagn 1 1 Fori of terraTn 
types E F&1 or ceil Images. [ Oj* Different types of 
texture will be discussed further In section 3,9, 
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Figure 3.1; The Intensity-Beaton Tree 



S(t) (white points) Threshold The Tree Structure 
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Chapter 3 ftftpro sent Ing an image as an I ntensity-^region 
Tree 



3. I The Bes k Method Used 

Consider an image, I, defined Cfl a rectangular 
raster of points, so that Up) Is the I Ight Intensity at 
the point p« for any given light Intensity "nresholC t p 
define p. zat of points SEt) = /p|Hp)atV, the sat of 
points of Intensity t or greater. tach of the eighl 
pictures ir figure 3, I (previous page) shows such a set 
of points, for some threshold* For any t t f he set SttS 
can be partitioned Into disjoint connected subsets R s (t], 
which will henceforth tie Called "regions". Thus; 

5<t) ■ R|( + >(J R J- < + J U' " -U R h ( + > * 
where R i[l^i" Q) ^ ^ J ■ ard Bacri R i ' s a connected set 
of points. Note that SCtj?CS( t, I if t a >t, t so each 
region at threshold t A must be a subset of so^c region at 
t . 4 The regions thus fall naturally Into a tree 
Structure baseion this subset relation, as shown In 
f i gu; r a 5.1 + 

Another particularly graphic way of looking at 
the tree Is to visual lze the intensity function plotted 



24 

in the form z"Mx,y)* SITclnp this function with a 
horizontal plane at severs! threshold levels, the tree 
Can be pictured as in figure 3,2* An. Intensity Contour 
map of the pear is shown rn figure 3.3 in order to show 
how the regfens are actually nested, 

3 *2 Qua nt f zat I on 

Choosing a set of threshold levels -(V ■ ^ ts 
equivalent to quantizing the tight Intensities In the 
Image,, in terms of the Information retained In the tree. 
The more threshold levels In the set, the greater The 
depth of the tree generated using these leve-ISj We will 
generally consider threshold sets which are evenly spaced 
J n the log of the ITght Intensity, although a tree could 
he generated from any arbitrary set of levels* Using the 
log Of the light Intensity generates a tree whose 
structure rengfns basically the same Tt the Illumination 
is scaled u p or down by a constant factor, 

3*3 Geometry of the Tree 

In the limit of a continuous tree C in yhleh the 

spacing between threshold tevels approaches z&ro) t the 
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Figure 3,2t The Region Flanes &hoWn as Slices of the Intensity 

F unc t ki n 



L i gh L Intensity 
s - f(x,y) 
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tips of the branches represent local maxima In the Image* 
Beg r nnlrig at a branch tip and movlnr? a I Ohg It fn the 
cirection Of I pner Intensity, the reoion expands from the 
max Fmmfi point to Include Other nearby points, assuming 
the Intensity function Is continuous in that area. Each 
tree branch can thus ha thought of as a growing region, 
A fork In the tree occurs whenever t w o or ncre of these 
regions ccmbJne, forming pne new larger regFon, In thJs 
case, t ho branch associated wFth the sub-region of 
largest area shall be considered the "main branch", and 
the other branches shall be called "sub- branches ". If 
the ortglnal Image 5s slightly noisy, then as a region 
"expands" fmovlng along a tree branch from high to low 
Fntensl^y), ft will engulf large numbers Of smaller 
regions which appear ahead oi It* advancing edge, 
resulting in many short sub-branches on the tree, When 
two regions of substantial area are combined, it 15 not 
really Important which is considered the sub-branch. 

The highest region, on the tree represents the 
brightest point In the image* If the threshold Is 
lowered far enough* aH of the regions will eventually 
merge Into pne region containing all of the Image points* 
this shell be referred to as the "root" af the tree, 
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3.4 Trees with Incomplete Region Information 



In the preceedlng dTjcusSien, the regions 
themselves have been co n s I d er ed to be the elements of the 
tree* Let us now consider an abstract tree structure fn 
which the elements of the tree are not the regions 
themse I v es ( hu1 nodes containing iMor nation about these 
regions. Such a tree shall t>e caUed an "Imase Tree", 
■f each node contains a complete description of tho 
region to which it corresponds tthet is > If R^Ctj) Is 
given for all I and t : ) , then the tree contains enough 
date to be able to re-construct the imago exactly,, to 
with En the limits imposed by the oua nt I zat ion * 

If each node contains only statistics cf the 
corresponding region, rather than g complete description 
of the region, then the tree contains less Information 
than the original i^age. These are the interesting 
trees, d&splte the fact that the Image cannot be 
r ccons* ructed from them. The problem of pattern 
recognition can be viewed as one of throwing away 
Internist Ion 1n a selective way* To go from a picture of 
on apple to the word "appfe" represents an enormous 
reduction In Information ( lf a picture Is worth a thousand 
words*)* 
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In general t the nodes may contain any arbitrary 
set of functions of the corresponding region* I i 
particular, trie ones which will be used are the position 
of the region's center of mass (x ct y i } 1 the. area A of the 
region, (i.e. the number o* points In It), and a measure 
Qt the second momonf about the Center of mass, called the 
eccontr tc Fty e. 
The eccentricity r s defined by 

\ / 2 2 

A <*^ • 

ell pts p 
3 n re g t on 

e is l,Q for a perfectFy circular reelon, and is larger 
for a more tlonca*^ r e <j f qti + 

Tha eccentricity Is a d lmens i on i ess e|LPBntFty 4 
whfch remains th* same ff the region size ts scaled up or 
■down* It represents a normal F zed moment ft f inert fa. about 
e line thru the region center of mass perpend lew tar to 
the region plane* It can be shown that no region can 
have an eccentricity Tess than i.q, end that any shape 
other than a circle has a higher eccentr f c rty. This Is 
because a circle has the smallest moment of Inertia for a 
g Fven a rea , 

for a I by f rectangle, the eccentricity Is 
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E = 



n v * 

which is 1.0*7 for a square.,, 1,31 for a 2 by 1 rectangle, 
and 2*23 for 4 4 by I rectangle. For a high elongation 

t t e = TTf/6. 

Note that this definition of eccentricity E 5 not 

the standard eccentricity of second order curve?. The 

eccentricity of an ellFptrcal region of semi-axes a and h 

Is 

e * I /a + jA 

Z ^b nj , 

which ranges from \ to &^. Tne normal definition of the 

eccentricity of an ellipse Is 



whlcn ranges from to I 



J 
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More cc-nolcx refilon statistics could be stored an 
the tree. If the x and y second moments are stored 
separately, then the "domlnent axis" thru the region 
center of mass can be easily computed* This Is a line In 
the pi arc of the r eg tor. points through which the regiart 
has mlnlnftum moment t Inertia, Higher moments could also 
bo computed, although their Interpretation Ifi terms of 
high-level shape descriptors is less clear. More 
complete shape de scr l ot o r 5 . such as the results of a 
Mad la I A* Is Tcsflsfom Zul could also be used. 
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The choree of more Complex shape deser iptors 
depends on the particular recognition tasks b e f n g 
performed. The simple statistics of area, center of 
mssSj and eccentricity can yield much useful Fnf ornat f on, 
however, and attention will be focused on th*m. It will 
be fieefi that they are quite useful for the analysis of 
Surface properties and simple shapes. 

1,5 Sub-pr-ograms- of the I ire fie Tree Systerr 

Programs have been written to obtain the image 
free of ei given scene, Measurements from g laboratory 
scene are read Into an array by an ima ce-d 1 ssec tor 
iameraj and a list-structure tree r 5 penerated* The tree 
can be printed out, showEng the parameters associated 
with each node, Programs also can graph against" the 
threshold any region statistic stored on the nodes, along 
Sorre path on the tree from a b^artr. h t I p to the root t The 
Original Image can be displayed, and sny arbitrary r e F O n 
can be shown su per Trnposetf upon it. For more detail about 
these program Sj see the appendix. 
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5,6 The Tree cf a Matte Sphere 



let us consider the tree resulting from an Image 
of a sphere with a matte surface* A matte surface 
exhibits b reflectance which is fairly uniforin In all 
d f r ec t E G n S r e g a r 6 \ e s S of the angle of the incident light. 
The Image of a sphere Is a circle. If we assume the 
reflectance to he completely uniform, and consider a 
Sphere II* from the camera position, then the Intensity 
as a function of radius r over this circle Is 

2 \n 

Itr} - Cl - (r/P) ] 
where R l& the radius of the projected circle, end the 
Intensity is normal i2edl to I at the central point, This 
formula simply expresses the fact that the projection of 
a surface seen by a viewer Is proportional to the cosine 
of the angle of the viewer from the normal to the surface 
(soft figure 3,4). Thus., assuming uniform scattering^ the 
intensify cf the light Is proportional to the COS Ine of 
the Incident {and viewing) angle, The intensity value 
actually read frgm the vidlsector i s t » C + SZLoc-tl), 
where C Is the reading at the central point t I Is the 
intensity, and the Log Is base 2. Solving for the region 
area as a function of the threshold t, we net 
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Figure 3.4: Formula for the ftef lectante of a Sphere 



The sphere i& Lit from the camera. pty$ it ir>n , 
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(tela timely fat) 
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A * EO - 2 3, 

where H is the ares of the full clrclti Its Image tree 
should have only a single straight branch, whose T I ft 
ccrrSfip&ndS to the central point. Each of the nodes on 
this branch represents a circular region centered about 
this point. 

A picture of a white sphere on a ^ I a c k b =] c k p r n 1 - 1 n d 
was actually restf Into the computer from the vidlsector, 
end a tree was generated by the jrtL-t-Jurc previously 
described, The t r eB had essentially one nafn branch, 
although there i-ere a fex v or y snort si, b-i?-*anchc s 
rep resent! ng regions of very snail area, which we re 
neglected. The measured" region area and the theoretical 
curve are plotted together In figure 3«5* 

Note that the measured curve rises considerably 
above the theoretical curve E n the central region, This 
Implies that the intensify is not I inear In cosine of the 
incident an<jl<>, but is serve* ha* convex,, as In figure 3,6. 
The sphere used for these studies had an extremely matte 
surface, and hence a negligible highlight. The sudden 
rise at the end of- the curve is due to the threshold 
lowering to below the Intensity of points In the black 
bac kground . 



Figure- 3.5: Region GsrowUi Eve a Sphere 
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Figure 3.6; Actual find Assumed Slit face Re f 1 ec ta rit e 
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5,7 Effect of t be Specular Component 

As was discussed in chapter 2, the reflectance c i 
a surface cen be considered to be a superpos It [on of b 
specu ler and a matte component. A mirrored sphere would 
give rise to a pure specular reflection, whJch would 
clearly be an image of the light source, plus a 
reflection of anything else In the room* If the surface 
Js rtot highly mirrored, this specular component wl I I be 
greatly attenuated, so that it can be netjleclec, except 
for the Triage of the bright I i ght source, which will h* 
significant despite the a ttenu-a t [on , Thfs reflection of 
the light source Is called a "n I ghl i ghf ", and will 
general! y be considerably brighter thun iho surround fug 
points. The magnitude of this hEghl ighl relative to 1 lie 
matte component Is. a measure of the specularity of the 
surface. 

Consider the effect of this highlight on the 
image tree, as-suninnj the light to come from a sma i I 
(nearly point) source* This * M I produce a small, bright 
spot on top- of th-o local maximum tn the matte component* 
As a result, a long section of the tip of tne tree wi ll 
represent a Sir.s I I region Of fairly constant area, This 
Is a result of the "spike" In the light intensity 
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function resulting from the sma H t bright highlight. 

Consider the set of spheres showi Fn figure 3*7* 
They were all painted *lth a n&tte white paint, and then 
coated with zero through jsven co^ts of clear* enamel, 
giving them varying degrees of specularity. A graph of 
the region ares vs, threshold (figure 3 , S ) shows the 
smal I flat section of the curve represent Inn the 
htghllght p for one of the spheres, Figure 3*9 gives this 
highlight depth h as B function of the nu Tiber of coals of 
lequer, Illustrating how. the surface speeu I ar I ty can be 
measure In a simple manner* The Irregularities In this 
curvei are probably due to the difficulty |n applying the 
coats of laquer uniformly* 

3,9 The Surface Convolution 



Local I yj consider a curved surface to be a part 
of a sphere of the sane radius of curvature, Accord In-g 
to classical optics, a spherical mirror has a focal 
length of one half Its radtusft, and will form a virtual 
Image of the light sou re e as shown In f fgure 3,10, If a 
light of diameter (! and distance L from f ha object is not 
too far Off the caroera-OD j ec t ex Ts f then the diRF-eier cf 
Its Image Is about 
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Figure 3,7: Specularity T«st Spheres 
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Figure 3.8; Illustration of mghllEfct Depth 



Graph l£ fo-E sphere 7 o£ fi&ure 3 h 7 



i.2 



Region 
Area 
(in po int s ) 




I r f P M I I i I TIT i i | i i i I i 

422 35B 374 35D 526 302 



Hi v esho 1 d 



A\ 



Pifiure 3,9: Highligjit Depth vs. Humfa&T of J^iqu-cr Coata 
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4 5 
cf * = d 



\l + 2UHJ 



if, s R -*. w, d ■ -► -d, as is indeed the ca se for a flat 
w'rror,} thus If the sl£e pf the tight source and the 
approximate distance of the object from the camera are 
known, the curvature of the surface con be determined 
near a highlight. Even If the 5 fze of the light source 
Is not known, this mothod gFves the relative curvatures 
■ f there are severs I different highlights In the scene, 
A good way to determine the &i?e of the source rs to take 
advantage of v cr 1 • '. a I i I y hv knowing the approximate 
curvature of some object | n the Image. 

Many surface? wl I 1 "smear out" the Inage of the 
1 r ght f resulting in a broader hlghltght then would be 
gutter, from a -1 i r ■:;: r cv s^rfr:cf; c. ' ? a„ i vt I t -r * cjrv^Lrfi. 
The highlTghT seen can be considered to be the 
convolution of the Image of the light source and the 
N Impulse response" of the surface reflectance, If the 
light source Ts a sufficiently small point, then Its 
image can be considered to be an Impulse, and the surface 
"smear" function can be read directly from the region 
area vs, threshold curve. 
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3+9 Texture 

"Texture" refers to variations in the light 
Intensity which, are very small In size compared to the 
objects being recegn lied* 1+ has two bas re C3u?es + 
"Visual texture" Is. due to variations In the reflectance 
of the surface, and "tactile texture" Is due "to mfnute 
protrusions or depressions supor Trapesed upon a basically 
smooth surface (the sort of texture one can feel wr-fh a 
finger J. If the size of the texture Is smaller than the 
resolution wTth which the Image has been samp I ed , the 
Intensity variations w I f l Bwraav out, and the texture 
hill have little effeM on the tree, aside from affecting 
the surface "smear" function* If the texture Is large 
enough to be d Iscernab I e, however, It will produce e 
distinctive effect on the tree. 

Texture Is a mu I t E-d (mens tana 1 feature, and "here 
are a correspond Tn I y large number of te*tura I properties 
whkh could be measured. We are not concerned here with 
producEng a complete description of texture^ but rather 
with detecting features which might be useful In mating 
an object Identification* Although such features can 
help discriminate between objects, they do not gEve 
enough | n format Ion to re-construct the texture exactly. 
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3.9*1 Visual Textu 



re 



Consider the two spheres, shown In figure J* f I t 
The spheres were pa fnted with a irette *h|te paint, then 
marked wtth red Tnk to produce visual texture-. The same 
two spheres ara shown In red, white, and green light* 
Sfnce the red Tnk Is hiohly reflective in the red f and 
very ab-sorptTve in the green, these Mahtfng conditions, 
produce light, medium, and heavy texture contrast 
respectively, with all other factors be inp held content. 

There are two kinds of texture, wEth respect to 
effect on the Image tree. The right sphere shows small 
d I sc o no ec t ed light patches on a connected dark 
background, and the left sphere Shows disconnected dark 
speckles on a connected light background. A. Mght spot, 
be ino a local ma* imum in the light intensity, will 
produce a tree branch. The node; on this branch wMl 
represent regions the size of the spot* and so will have 
very small area. The length of the branch will depend on 
the relative brightness of the spot compared to Its 
neighbors, since whan the threshofd reaches the Intensity 
Of the neighbors, the region corresponding to the spot 
will be swallowed up by the larger regfon surrou r-d Intj It* 



Figure 3.11; T :■]■■: t .ir« Test Spheres 
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Light speckles wt II thus produce a large number of sub- 
brenchss whose ienpth represents, the Intensity of the 
speckle, end nhcse "slie* (the size of the corresponding 
reoions) represents the size of the speckles* The tree 
corresponding to the light -speckled sphere photographed 
In the green light {deepest texture) Is shown In 
figure J. 12. Note the many branches produced by the 
sp ec k 1 e s » 

The number and length of the sub-branches 
provides a measure of the degree of contrast of the 
texture* Tbese Quantities are shown In figure 3 T 1 3 for 
the I Pg ht- spec k I ed sphere under the three \ Ightlhg 
conditions Note how th#s« quantities thus provide en 
Index ef 'exlu^e Contrast, Just as the highlight depth 
and surface smear Function provide an Index of 
specularity. Information about the details of the 
texture can also be. obtained, up to the limits imposed by 
the particular shape descriplors used on the nodes of the 
tree, Round speckles will produce regions of low 
eccentricity, whereas streaks wH| produce regions of 
very high eccentricity! If the direction of the dominant 
axis of the region were recorded (corresponding to 
record fng the second iiMfnts In the * end y directions 
separately), the dominent axis of th* streaked textura 
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Figure 3,12; Tree of the Light -speck led TuxtuTC- Test Sphere 

tgreen Light) 



All suh-branch nodes represent regions of small area* 
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Figure 3*13: Hnpber and Average DLipth of Sub-brsnches for the 

Light -Spelled Texture Teat Spheres 
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could be determined as wall, 

Dark speckles will have a different effect, 
however. Since they are local Minima In the Intensity 
function, rather than local max rma ,. they will not produce 
branches on the tree, but ra+her will produce holes In 
regions, This is shown by the tree of the dark- speckled 
sphere, shown In figure 3«I4* The only effect of these 
smell tiol es Is to raise the eccentricity of the growing 
region, as shown In figure 3 . \5 t which shows the main 
branch eccentric \i y vs. regfon area for the dar H-£ p cc k I etf 
sphere In the three different colored 1 lights* Since the 
eccentricity change fs 50 sma I f f these three curves can 
he compared [r, this way only because all factors except 
the degree of texture were held absolutely constant - the 
same sphere was viewed from exact I v Hie same camera 
position and with exactly The same light source* Nothing 
was moved; only the filter over ttie I Tght was changed. 

The difference between the trees for the dark 
speckled and the I fght speckled spheres (figures 3« 12 end 
J k 14) exposes a basic asymmetry \n the Image tree wfth 
respect tc light and dark. This asymmetry Is net just 
Confined to texture, of course* Locally bright areas 
will always produce regions and hence tree nodes, while 
locally dark areas w I I I always produce holes In regions. 



Figure 3. 14; 



Tree uf Lhe Dark- speckled Texture Test Sphere 
(green light} 
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Figure 3.15; Eccentricity vi. Re&ion Ar^ for the 

Dark- speckled T^xtnTt Te-st Spheres 
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altering til* statist its of nod cs that would otherwise 
ex Fsf anyway. 

The tree could easily be extended to find dark 
speckles by generating an "Inverted" tree for the area 
Inside each region* hr\ Inverted" tree ts a troc In which 
the r eg tons- rep resent image areas less than threshold., 
instead of greater than or equal to. This will b-e 
further discussed In section 5.£.2 t 

3*9.2 Tact Tie Texture- 

Small bumps an the surface of an object 
essentially produce many tiny "micro-objects 1 ' with tne 
same surface properties. If the s Tzc of these is below 
the resolution o+ the lrnnc ie^pMro, the effect wilt be 
only on the surface smear function, If the texture is 
larger than that, and the surface Is talrfy specular, the 
result will be nany tiny highlights, producing the 
equivalent of a light-speckled visual texture* 

3.10 Shape 



The Nane + r ce carries shape Information in two 
ways; In Its forrij and fn the behavior of the region 



StaMstFcs stored a long 1 + s benches. The I n t erpr eta t T o n 
in terms of object shape of the simple region statistics 
dEscus-sed so far depends upon the object being simply 
shaped > since tho eccentric Itv does not give cnouch 
Information to distinguish between different cenp I Cx- 
shaped rcgFons. Nevertheless, much useful shape 
Information con be obtained even with, very simple 
statist 3c s, particularly in a rcccgn ft I on -or tented 
application In which, there Can be restrictions on the 
shapes, considered, 

3,10,1 The Main Branch 



Consider the object shown In figure 3,16, Its 

tree Is a single main branch, Just as I n the esse of i 
sphere (a crude contour map F s 5 how n En fio/ur* 3 t 1 7 ) , 
The simplest Indicator of rts shape Es the eccentricity 
of the en+ire object, which Is about 1.4, clearly 
Indicating It to be quite elongated* The entire curve of 
eccentricity vs, threshold it shown In figure 5»I£J, The. 
flatness of this curve ind Fcates that the region probably 
doesn't change Its shape very much as tt grows^. and that 
it has a smooth surface with np sEgnFficant 
Irregularities. This Fs not a unique Interpretation of 
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Figure 3.1*; h Matte-wts£te Paint *rl Squash 




Figure 3.17; CDntour Hap at t\\& Squash 
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Figure 3,L&: EteeTiteicity Curve of the Squash 
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the curve, but ts a reasonable inferente given the 
assumption that the object Is not highly IrranuPar. The 
tump in the eccetitr ic f ty Curve at the bright end ts 
typical of a small newly developing region* Since the 
5 I op & of +hc llgh+ Intensity function is very small near 
a I oc a l m a x i m u m > a small reg Ion about that paint will 
tend tq have jagged edges, and hence a high e.c.c.en +r i c i t y. 
As the region expands, the intensity gradient at the edge 
Increases^ so the edge becomes stralghter, and the 
eccentricity Is reduced. 

Consider the plot of added region area, shown In 
flours- 3.19. This quantity shows the excess area added 
to a region above the sum of the areas of tts sub- 
regions. Since the intensity neasured Ts a monotonfc 
function of the angfe of the surface to the camera, the 
added 1 region area is the projected area of that part of 
the surface on the object *|th a particular slopei A 
bump In this curve represents a large area of relatively 
low curvature* The only one In this case Ts near the 
highlight* 

Figure 3,20 shows what the area added to a region 
looks like - It is the area of a region minus the area of 
elf Its su fc-r-ug icn s. Note thut the statistics u sctf are 
such that from the statistics of a region A and those of 
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Figure 3.L9; Added Region Area Curve of th« Squash 
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Figur* "i. 20: I [.Lustration o£ an Added Avea ftogton 
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a sub-region B, +nd stat fst les of the dlf ference A-E can 
be computed. {To compute the eccentr Ic My of ronton A-B* 
the eccentricity, region area, and center of tiass 
positron of regions A and B must all be known*) 
Computing Information about the shape of such a 
difference region gives Information about bulges 
deve loping in a rag Ton, direction of motion of the center 
Of ma&s, end Other properties of ill those points on the 
surface within some giwen range of Inclination to the 
cam era * 

The added artt curve would have t*o peaks for tho 
hypothetical object shown \n figure 5.2 1, due to the low 
curvature of the annular region indicated. In this case 
the eccentricity would be constant at 1*0 and the tenter 
Of mass positron would be stationary, s"rce the rerrlcns 
would all be concentric circles dye to the rotational 
symmetry. For the pear-like object In f Inure 5.22* the 
protrusion *ould also Increase the added area curve, but 
\n this case* the eccentricity would Increase as well* 
and the tenter cf mass would shift. 



Figure 3.2Lh A Bynmet rica L ■Object with Two Added Axca Peaks 
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Figure !5.22; A Contour Map of a Hypothetical Object with a 

Protrusion 




1 ' Protr ufl io n." 
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3.10.2 Sub-bra nc he 5 

Protrusions of the sort Illustrated in 

figure 3.22 wilP gften produce si^i it icia-nt su h-bra nr. hr; & 
on the tree. Tho meaning of a sub -branch must be 
Interpreted in conjunction with the I rvt prrngt 1 on stored on 
it # and on the main branch to which It attaches^ The 
attachment of a protrusion region, for example, will 
generally produce a rise In the eccentrEeltyof the ma in 
re-glen, and a shift In Fts center of mass. The possible 
Interpretations of a- sub-branch defend very heavily on 
thft part feu tar Identification for which th* tree is being 
used, A discussion of the r n terpr eta t l on. of shape 
Information for a particular set of test objects will be 
g iven 1 rn sect Eon 4.2. 

3.10.3 Non- interference of Texture with Shape 



Figure 3.23 shows graphs M the regton area for 
the speckled spheres of f Igure 3 h I I ,, norma I (zed to the 
light Intensity, These graphs Illustrate that the basic 
shg pe-d CSCr i b I no paran-eters s r e no* affected by object 
texture in a significant way. This fs basically due to 
the averaging nature of the region descriptors used. 
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Figure 3.23; 



Region Area Curves for the L3.K,fot-spet:kle.d 
Texture Test Spheres 
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T h T & Insensibly Ity to t e n t ^i r a T rnterference f 5- ■? 5 r e g t 
Improvement over- nost previous methods used o rt curved 
objects, such as Horn's qts I y" 1 " lea I method, which Is 
ccunp lete I y useless in t ha prcjenCE o * textura, Ldgd- 
flndTng methods are a I so confused &v sharp texture. This 
advantage f s vurv i-nocrtaiit in the reeogn ft 1 on of real 
objects^ 55 will be seen in the next chapter* 
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Chap** 1 * A U se on Real Object s 



4,1 Pru n inn 

Regions generated by smooth objects *lth smooth 
surfaces should fn theory always have SmOath boundaries, 
in an actual Image, however, minute surface fluctuations 
a nc nol&a will cause the edge of the region to be hfghly 
irregular. It the irregularities ere great enough^ Small 
sections of the region will be detached; that Is, th«y 
will actually form separate snail rec I or s, Since the 
area separating these Sffra ! I regions from the edge cf the 
nearby large rag ton Is only slightly dimmer than the 
reg ion po In +s, thn^fi swa I I ren lens will join the main 
region at a threshold only slightly I ower thani that at 
which they started. They wi M thus praduee very short 
branches on the image tree, whose regions ere o* gma I I 
area* these regions are es gan t fa H y artifacts of the 
partTcular levels at whtch the threshold Is placed, and 
thus have no particular s f c n I f 1 eanee-* In order to avoid 
the waste of sp-ace and t fme needled to store and analyze 
these branches, they can be h pruned 1 ' away ss ^he- tree fs 
generated. This Is done sfmply by removing branches 



66 

w hitch are shorter than a- f fxed length In Intensity units 
and whfch a t so represent recTofls smaller in area than a 
1 ffc&d -5 ire* 

Texture also produces short, small tranches, so 
these thresholds must be adjusted 50 5 5 not +0 throw 3 way 
too much. The goal of pruning Is to eliminate all the 
really tiny thresho Id-art I fact branches without 
el r m 1 n a 1 1 n g branches which represent texture. Since the 
artifact branches will be only one or two threshold 
levels deep, this can be acconp- I I shod by etfjustfng the 
spacing of the levels so that one level represents a 
smaller increment o* intensity then the minimal Intensity 
texture to he considered. 

4»2 Some Sample Trees 

Various types of fruTt wl M b& used as exomptes 
of fairly simple real objects with smoothly Curved 
surfaces, In order to net a feel for how the Image tree 
behaves on fnterestTng images. Two sample Images will 
now be d iseussed to give a general Idea of what the trees 
are I E ke, 

Consfder the apple In figure A t \ t Starting at 
the brightest node end plotting the region area down the 
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Figure 4, It Apple 




70 



tree ta the root gives ffgure 4,2, The sms) I flat 
section of the curve from 3S4 to 356 Is caused by the 
bright hlghl Ight* The regTon arefl then begins ta grow In 
a manner similar to the sphere previously discussed. The 
growth lepers off as the region expands to fill t^C 
object* Then, at about Intensity 216, the region breaks 
out of the object Into the- background. 

The regfon center of hei ss Is shown In figure 4,^, 
It stays at the hlghl Fght for a whl I e, then shifts to- the 
left towards the I Tt side of thr* ap?le. It then slowly 
moves to the right towards the center of the apple, as 
the f eg Ton grows out to the app I e's edges* The 
eccentricity (figure J t 4) starts off faTrly high, but 
rap Ed 3 y reduces towards circularity, nearSy reaching 1,0 
when the apple has f F I I ed out. 

h portion of the tree, displaying only the region 
a< re a paranflf er, Is shown In Hgure 4*5 + Although the 
tree Is fairly simple, ^ significant region fs. added on 
the the main branch at level 2B6. The center of mass 
shows this ares to be above the main region, and Et \n 
fact Is the brTght ref lection from the back of the stem 
hoMow on the top of the a p p I n , which can be seen in the 
photograph 1n figure 4 . I . It Is very common to hove suth 
a branch on the trees of fruits with stems, and higher 
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Figure 4,2: Region Area of Lhe Apple 
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FigLirft 4.3: Region Center of MaiS of the Apple 
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Figure 4,4; E«entticity of the .Appl^ 
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Figure 4.5: Tree of the Apple 
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level recognition routines could take advantage of this 
fact to help find stent areas. 

Mow consider the pear shown fr Hgure 4,6. Its 
tree, shown In figure 4 + 7 p Is topelogica I I y similar to 
the tree of the appfe, Including a snail sub-branch with 
significant area. The graphs of the varfous parameter s„ 
however, shown In figures 4.&, 4,9, and 4,10, reveal that 
th [^ sub-branch ha s a different f nterpretat Ion than In 
the case of the apple. FTrst^ Its center of mess shows 
It to be positioned to the left of the main regFon, 
rather then directly above Ft + Second, at the point et 
which the t w o branches joTn, there Is a r f se in the 
eccentricity in the case of the pear, whereas there Is 
mot [n tho case of the a pole. Finally, the cccent r 1 c It y 
of the apple Just before breakthrough Into the background! 
was near 1.0, whereas the eccentric itv of the pear Fs 
about I .2 f which Is s Tgn I f Scant I y higher. Information is 
also avaFlabfe concerning the surface properties of the 
pear. The pear's highlight shows 3 wider "Impulse 
response", which Indicates, that its surface, although 
somewhat shiny, is not as highly specular as the apple. 
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Figure 4. 6 i Tear 
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Figure 4.7: Tree o£ Feet 
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FifiurC 4,8: Center of K3S3 of the pear 
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Figure 4.9: Eccentricity of the year 
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Figure 4,10: Region Ar^a of the Fear 
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4.3 Useful Features for Fruit Recocn 1 1 ion 

Wo will no-w attempt +o list some +es+ures which 
can be easily extracted from the Image tree, so that the 
c lass itleat Ion of fruit may be systematized* ThJs Mst 
Is rot Intended to be exhaustive. In fact, quite to the 
contrary^ It Is Intended to show that recognition of 
fruit fs possible *r T t r* only a fen very sfirple features* 

d*3+l A Sarrple Set of FruTt 



In the course of studying the Image tree method ^ 
a large number of fruit were processed to study The 
effects en real Images, In addition, g large number o-f 
fruit were given identical orocesslng under Identical 
conditions one day fn order to gather some statistics on 
the various features which can he extracted* Photographs 
of the fruit In this samp ( e set ere shown En figure 4*M t 
The fruit used were Bartlett pears, Macintosh apples, 
sweet pears, and oranges. The test Irnocas Include tfvo 
views each of the Bartlett oears for a total of 25, two 
views each of the apples (total 10), three of the sweet 
pears (total 15), and one each of the oranoes, Three 
taped Images of peaches *^^> also Included In tha sample 
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set, although they were record ed under different 

e ircum stances, Peaches were unavailable at the Tfme the 

sample sejl' hn s rur, 

4. 5 ¥ 2 Specu (frr i tv 

As was discussed in Section 3.7 P the "impufEe 
response" of the surface can bo approximately obtained 
from the region area vs. -threshold curve at a branch tip, 
He would J I ke to characterize this curve In order to 
extract some significant features that are useful for 
recogn It I on purposes, One way to do this Is shown in 
figure 4, 12, At the branch tip, the second derivative cf 
the re c. Ton area curve is por. itrv^ due to the specular 
component, but negative due to the matte component, h 
straight line fitted to the curve at The inflection point 
Is shown, extended to Intersect the axis* T he 
intersection point is caired the "matte intercept", The 
value of the curve above this intercept F5 used as a 
measure of the width of the surface function, as shown on 
the figure* It is called s, for the highlight "smear" 
width, 

Ano+her measure of the surface function is the 
j-ip I lludc o- 1 the hip- i(n+, u ! s<j rrnr <cc " i t nc f Igure. 
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Figure 4-12: Cha raster la £ng the Region Area Curve 
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This Can ba measured Tn various ways, but is here 
measured as the amplitude af the h3phl Ight above the 
matte intercept. 

A scatter diagram of the smear width s vs. t"ie 
highlight amplitude h is shown Jn figure 4.15. Note that 
the peaches, apples, and" oranae are separated very well 
by thoir highlight properties, but that the t*o types of 
pears not finly have similar properties, but also sho* s 
very high degree of variation in these parameters. This 
Ts partly because their surfaces arc rather lumpy and 
uneven, which disrupts the highlight region. As will be 
seen later, this unevenness can be used to help Identify 
t n c 'i! . 



4.3.3 Simple Global Properties 



Two very simple properties of a fruit are its 
brightness end fts size. These are both properties which 
are useful only relative- to sons additional \nf crmat ion 
not contained fn the ifnage alpnpj specifically, the II g-ht 
intensity and the object's distance from the camera. If 
this Information is available, these two features can 
contribute recognition Information. These quantities can 
be obtained, In many cases, from other Known objects 1n 
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Figure U. 13 1 Smear Width vb. Highlight Amplitude 

for the Sample Set 
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the Emage + In the experiment described nsxt, the sample 
ffuit ware a I I v i ewod with the so™ie light Intehslty and 
at the same d Isfance fron the camera, so that the fr 
intensity end size are comparable. 

The brightness of art Object is taken to be the 
Intercept of the straight line a pp rex i mat Jon to the watte 
component with the line of zero regiion ar&a f thu$ 
estimating the brightness of the surface If there were no 
highlight. The overall area is estimated by scanning up; 
from the roof of the tree until the first local minimum 
in the slope of the region area curve is found. The 
region erea of this node Is taken 35 the object '5 
projected area (see figure 4.12)* 

A scatter diagram c+ 1 ~r e se two quantities is 
shown In figure 4*14 for the sample truH. They a r e 
clearly not very useful fei* distinguishing between the 
fruit in the sample set, They would be very helpful it 
very lergc objects such as watermelons were lrcluded, 
however , 

Another optical feature which could be used is 
color* which would be very powerful far fruit. This 
feature was not studied In our experiments, because the 
processing of d Iff arent color Imagas of the game object 
would have added complexities end delays without much 
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Fi£vre 4*14: Brightness vs. Overall AiCa for the Sample Set 
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added understanding Of The Fmage tree. 

-Ui^ Oy e^a I I Sbape 

Our simplest shape descrTetor Is Just the 
eccentricity of the entire fruit outline regton, whrch \ s 
shewn plotted with the highlight depth En figure 4*15* 
ThiG parameter alone wfll [dentlty a banana t which has 
not been Included in the sample set. Note that oranges 
and apples are extreme \y round* 

4,3,5 Sub- branch Types 

So far, we have used only i nf or mat Eon extracted 

from the main branch. Mpny properties of an object 
produce sub- branches. |n understanding an imaoe we- must 
figure out what these Sub-branches represent, Some types 
of sub-branches will now be discussed* and a simple sub- 
branch c I ass Ft ica t Ion algorithm presented, 

4,3.5.1 Tact He Texture 

The orEsnges tn the sample set supply good 
examples of tactile texture, A cIose Examination shawS 
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Figure 4,15: 



Object. Eccentric ity ve. Highlight Depth 
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TheTr surface to be covered by small bumps and valleys. 
Since The surface I5 also highly specular,, this 
gralnlness produces nvri ad small h 1 nht rghts, as discussed 
in sect Ton 3.9,2. These produce small short branches on 
The tree. Te*tural branches represent renlons of small 
ares, and arc near the t fp end of the tree. The number 
of sub- branches en a tree identified as textural by The 
classification algorithm sha I I be denoted by the 
variable T . 

4.5*5«2 Stems 

The Bartlett pears show (arre, long, lipht- 
colored !t«i$, The branches produced by these stems are 
easily identified by thair sr*a r 1 size and targe 
eccentr [c Ity* The number oi stem branches la denoted 
by S. 



4*3*5*3 Protrus 1 



5 Ions 



A pear Is basically a spherical shape with a 
protruding bump* These protrusions will frequently 
produce* major syb-bren-c^i en the tree; as In the case o* 
the pear discussed In section 4*2. Such protrusions 
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generally have a larfle are? t and usually produce a 
significant junp fn the accent r le 1 ty of the main branch 
at the point whore trtey join ft. The number cf 
protrusions will 6b denoted by the letter P lusually 

?r I J. 

d,3-,5 + 4 St am Hoi lows. 



An ap.pl* has a somewhat con fca I depression on top 
Tn the Spot the s*en- Is attached. The stern itseM \s 
sna I I er and darker than In the case of the pear, Thfs 
stem hollow wl ll often produce Si separate branch on the 
tree, ss the light reflected from the. back of the hollow 
is surrounded by darker points on th« riir of The he I low. 
Furthermore, the dark stem will often bisect this region, 
producing two sy b- branches. Thus a slgnltfcsnt sub- 
branch which causes a cro? In the na 1 n branch 
eccentric Itv when It joins Is likely *o be a stem hollow, 
end this Ss reinforced H there Is another sinllar regfgn 
nearby, The number of stem hollow recfons is denoted by 
the le*Ter =■ Usually 0, I or 2). 
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4.3. 5 + 5 Surface Irregularities 

There are frequently a number of branches which 
do net fall Into any of the above categories. These 
often are due to- Irregu I a r it res in the surface of the 
object* These irregularities are lerger then what r 5 
called tactile texture, but smeller than those large 
enough to be ca I i ed protrusions. The number of sirch 
branches shall be denoted by the letter I. 

4,3,6 Suh-tranch CI a S£ if ica t [ on 



A very simple algorithm was written to classify 
sub-branches. It Is shown Fn flow chart form in figure 
4.16, The parameter A represents the area of the sub- 
branch just before It joins the main branch. The 
parameter At fg the change In the eccentricity of the 
main branch at the point wnerc the sub- branch joins, Ac 
Is positive if hhe sub^branch produces an increase in the 
eccentricity, and negative If It produces a deer ease. 
The parameter j telrs whe-c or. the main branch the sub- 
Drench is attached, on a scale from 0*a (matte Intercept) 
to 1,0 (full object). It *he sub-branch j&lns the main 
branch In the highlight region (above the matte 
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F£ell t * 4.1&: Eub-btaDtbj Classification Algorithm. 

A ■ sy^-tranch a*ea [enLev 

i« ■ m-ain branch eccentricity 

chsfige at join 
j = point of join ae a fraction 

between OhO (matt intercept) 

and J.0 (full object) 

C J a a s t 
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? - protrusion H = stem holir>w X - te^re I ■* irregularity 



interf ae el 1 w* set J +o 0* 

This algorithm was ha nd -tompu t ed for each of the 
'fruit In the sample^ The flow char* fs recommended 6v«r 
the following very Sketchy description of the algorithm; 
If the btggest region represented by the sub-branch I 5 
large and Increases the eccentricity of the main branch 
significantly, call ft a pro+rusfon rag ton P* Otherwise, 
if tt Increases the eceentr I c 1 1 y only slightly, and Is 
not too small, it is an Irregularity I, If ttiere Is no 
change In the main branch eccentr fcHy f or if a small 
posftlve cnange Is produced by 3 sma I I regEon, than the 
sub-branch is either texture T or an Irregularity I, 
depending on whether 1t Is towards the tip or the rogt of 
the tree* Final ly, H the region Is large enough and 
produces a drop h 'he n*aln branch ecc entr lc Tty, and Is 
not at the tip of the free, If Is u steffi hollow H, A 
small region will be accepted as a stem ha flow If there 
is another stem hollow nearby, 

4.3.7 Object Recognition 

The flow chart of -figure 4 + l7 produces an object 
jt; i.:nt T 1 Icet Ton nnce the Su b- tranche S have been 
character f sect . In essence, oranges are Identified by 
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Figure 4-17: Object Identification Algorithm 
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havirvg lots of texture branches and belnt? very round, 
App les show stem hoi lows and are very round- A stem area 
identifies a Bart (ett pear immediately- Th « +w o +voes f 
pears are sorted out on the basis of their eccentric ity b 
the number of protrusion branches, end the number of 
Irregu | ar i t tes + Rouna objects with assent tally no 
highlights are peaches. 

The flow-chart shown correctly identified all of 
the f ru 1 I" with the exception of one BarMetf pear <EPIO 
whtoh was identified as a sweet pear. The pertinant data 
for each of the sample fruit are shown in figure 4. IB, 

Our conclusion Is that recognition of Imaaas of 
single fruits Is relatively easy, using the image tree* 
The Image tree allows the easy extraction of enough 
information about surface propert les,. shape 
Irregu lar It Eos, and general shepe, as weH as helping to 
spot specific characteristics such as stern hollows and 
stems, and the procedures which extract this information 
are reasonably simple* Kore complex routines which take 
the trouble to look more closely at the tree T s statistics 
should be even more reliable. 

The recognition procedures d!e scribed would be 
di Erupted (&S would many others) by occlusions* Shadows, 
missing stems, and object positions which hide 
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srgnff leant features, "any of these problems could be 
eased by a sultibfe vertical system, which could use 
other knowledge to explafn and correct changes in the 
Image trae + Other problems can be solved without higher- 
level aid, simply by making the recognition routines more 
clever. For example, occlusions ca n generally be 
detected 1 by the way tn which tw.O regions connect. Once 
en object is known to be partially occluded, correct Fons 
can be made to lis raolnn statistics whSch g I v* an idea 
Of Its f orriij under the assumption that the visible and 
the hidden parts or* similar. 

Even (n the presence of severe occlusion 
problems, the tree stiM gives valuable focal E nf crmat ion- 
about highlights and tex+yre. ^Ithouch the stems gave 
significant aid In identifying BartFett pears, the stems 
were not seen |n tei of the test cases, yet nine o* these 
were correctly fdenllfied* 

4*4 Faces 



This section, Illustrates the behavior of the 
(mage tree produced from a more complex smoothly curved 
object; a human *ace. It Is included to show another 
example of a real recognition task for which the Imaoe 
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tree is potentially useful. ft tree was penerated from an 
[m$(]$ of a face, seen full face and I it from the front, 
Tftis tree Is shown In figure 4.19, Branches of the tree 
have been labeled with the Iocs' maxlmi on the face to 
Which +hay correspond, and the shapes and positions of 
these regions Is shown In flcure 4.20, These regions 
might be useful for face recognition, at least for the 
simple angle of view and lighting considered hers* 

Contour maps at a single level of the tree are 
shoxh Tn figure 4.21, for each of two levels {marked In 
figure 4,19)* At level 3 13, most of the major regions 
seen In the photo appear, wfth the except tor of the lower 
lip highUght, which 1 s tons rderab I y d Trnmer , The contour 
map at levef 266 Is rather Interesting. Consider not the 
region Included within, the contour, hut the area 
excluded. This includes most of the mouth, the eyebrows, 
the eyelids (the eyes are closed), the nostrils, and a 
shadow area on either side of the nose. These are 
locally dark ^r^as In the, imago, These could Be Isolated 
by making an inverted tree - that is, by meMng. a tree 
With the Iner e negated. These locally dsr\ aregs are 
probably better places to begin face location, since 
there tre fewer of them than there are locally hrlght 
areas, and they are mora prominent. Indeed, there are 



Figure 4.19: Tree of a face 



I CM 



is'uiiiljer gives 
Teg ion area 
in points 



The Tt-giona Correspond in g to 
Che boxed liodes are shown in 
figure 4.2G 



threshold 26fl 



S5£l 
-+- S3S3 



lover 
lip 



left, 
cluck 



nose 



* * *** 



**- 



+ * 

7201 50 

* # 
C &2 B 3if- 

* • 

* * 
61S2 1M 

56b9 

under & 3$ £ 16 

nostrils * * 

H13 10 



i»J[ip 



103 3 between 

* * ** . e yfrbrowe 



515 

* 

-git 

* 

l 



s at* 



12G 

-7E- 

31 

a 

1* 

* 



1— 

il 2ss 



40 

* t 

254 — -S — 192— [HI 

* * * 

10 3 5 

« * * 

1 1 37 

* * 

1 1 



1632 
* * *■* ± 

+ 
70 J Tight 
* cheek 



BG05 

7750 

+ *• * * * * * 

7 ■' : 2 :■ 5 R 



121 

- 

Mi' 



6£ 

SI 

*■ 

■lie- 

* 

+ 
2 



3 07 



— threshold 31 j 



!0Z 



Figure 4.20: Seme of the Region* of the Face Tree 
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Figure 4,21: Elites of tt ( e Tree at Two Thresholds 
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■M per TfllQnt S which Indicate that as babies learn to sea 
fac.es, ttfiey f i-sT fixate on the pair of eyes C ',&!]» Qr*ce 
e face Fs roughly located t Pitcher I eve f routines can make 
serse of the locally brTaht areas with less difficulty* 
Figure 4,22 shows a contour map with both levels 
su per imposed ,, with the dark regions shaded. 

Note that the Image tree can easily be used to 
isolate facial features end determine their approximate 
position. In order to better characterize their shapes, 
more complex shape dascrtptors would prcbabiy &<* needed 
Than those which have been used so far. The imaoe tree 
Can be used to characterize the shapes of objects, such 
as noses, which have no "herd edge" boundary. This vlll 
be further dlscussod in section S*2 + 4. 
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Figure 4*22: 1U<e two Contour Maps Supe? imposed 
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Chapter 5 OFscussion 



5-. I Comparison wi-i Previous Work 

The lmane tree cai new be situated among the 
pattern recognition methods discussed In chapter 2, It 
is a "regions" method , rather than en edge detection 
*;■: ■;■■■ ', nrc: or- -i :.■ rl i f ' c: - o • + ■■"; - i c- or othc D r i.- - 
processing Of the Image* It extracts- Inf prmat ion about 
both the surface properties o# an object and about (ts 
shape- It does not require any high degree of precision 
of measjreirert wMh regard to the exact location of 
specific points in the image, and does nut make any 
fcssentlar use of perspei: ■* 1 ukt lr f : r ir a t 1 on . It do&5 not 
attack p rob J em s of the v parsing"" of an Fmage into its 
component parts directly, although Tt may aid this 
process by the wb\ It organises the ime ge Information! 

5- 1 2 Advantages 



The Fmage tree has a number of advantages for 
pattern recognition over many previously used methods- 
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5.2. I No Edge Probleir? 

By Virtue of be fno, a "region-*" method t The Ima^O 
tree Is Immune to many of the problems which beset edge 
detectors, These include false edges and gaps In real 
ed pes . 

5«2»2 Noise Immunity 

The use of mgments and averages a* parameters On 
the tree results 1 r* * system which does not re Quire nrost 
precision of measurement, and which Is insensitive to 
noise end distortion In the Image* 

5.2.5 Obtains and Separates Surface and Shape Information 



The tree also carries Tnf urmat len about both 
surface properties and shape. The two forms of 
in format Ton are well separated on the tree, with the 
surface Information being carried 1 at branch, tips, gnd the 
shape Information appearing further down the branch. 

Indeed, the tree can be thought of In a general 
way as carryFng detail r nf orm** I en near the tips of 
branches, a nd lower resolution Information Towards the 
root* For an image contain I no many objects^ the root 



toe 

will represent the entire scene, and various sub -branches 
w i I I represent subparts, and then sub-parts of The sub- 
parts, The tree can thus be thought Of 3S providing a 
range' of m ea stu r em? n t r ot differing degrees of acuity* 
These notion? of pattern recognition as a sort of 
"measuring 111 problem are due to Klrsch. 

5,2,4 Objects Without Boundaries 

The Image tree is easy to apply to the 
recognition of sbjeds without rea t edges or well-defined 
bou ndar le =. , such as a nose, or an object I it so that one 
side fades off gradually Into shadow. Assuming the 
Object produces ;.i separate tree branch, r t can :: e 
analyzed from the data at the tip of the branch, working 
down towards the base until the parameters indicate that 
j-he region Is taking Tn too much extraneous area to bo 
useful. Thus some' information etoout a nose can be 
extracted even though it has no well-defined upper 
boundary > because It has we I l-def 1 ned lower and side 
boundaries, Th I » simple task can be rather complicated 
for ed ga -or tented procedures, or for programs which are 
regions oriented but which do not make a series o* 
related measurements st different levels, as In the tree. 
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By the same arguments, the tree Kill contain rnforiltttlon 
about a smoothly curved object even ft It Is partially 
obscured, provided Ft contain^ a local brfghtness 
maximum. The procedures which analyze the tree must be 
able to detect the occlusion and to try to conn en sat a for 
It, 

5*3 Problems 

The separation of coarse and fine Information r 5 
not always maintained by the tree, yn f ortu nate I y* When 
branches representing two dffferent objects merpe, 
Information about those parts of the object nnt yet 
tilled out by the region may be lost. If a small 
highlight area Is swallowed up by a large** region before 
aehfevlng much depth In rts own right, the Information 
that would have been obtained about the local surface 
properties of thai erca are. swamped out. Wien s region 
representee son? object In e scene joins with a larger 
region representing the background, the reformation about 
the smaller object Ts lost. One case in which th [ s can 
occur fs when a dar* object Is or. a light background, or 
near a lighter object. Or, B | t er na t | ve 1 y , a region may 
extend beyond the boundaries of an object on one side 
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before reaching the boundary on the other side, possibly 
due to an overall greet Sent In the light Intensity* This 
snail be referred to as a "breakthrough". Although It 
ca n usually be easily detected by Its effect on the 
region parameters (sharp rise In the region area artrf 
eccentricity, and sudden shift In the tenter of mass)* It 
Still means a toss of Information about the side of the 
Object which the region has not "filled". 

5,4 Further Considerations 



5.4. ! Other Statist les 

So far region shape has been characterised by the 
region area, eccentricity* and center of mass position* 
There ire many other region statistics which could he 
used to characterize + ha regions, depending upon the 
particular recognition task at hand. 

One very simple addition which could be made 
wou id be to compute the x and y second monents 
separately, so that the major axis of the region could be 
found. This ts the axis about which the region has a 
Minimal moment of inertia, This would allow the tree 
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parsing programs to check highly eccentric reqlons toi 
proper or [^ntstig-n, given seme hypotheses about what the 
region represents. Other higher moments could also be 
used, although their interpretation *ou I d be sone-what 
more d 1 i t leu 1 1. 

It would be very useful to kno* I' a region has 
any major holts fn It* It Is very easy to tell i + it •-. a <, 
any holes st a M by ca I cu fat 1 rig Its eu I cr number while 
the connectedness of the re g Tori is beTno verified. This 
number can. be calculated on a local basis, us Inn 
procedures reported hy Gray f ^ t £.tnce the euler number 
gives the number of objects minus the number of holes, 
and since the region is known to be one connected object, 
the number of ho ' 6S In It is one minus the euler number, 
U nf ortu na re 1 v> this Is the topological number of holes, 
rathe-* than the number of holes of large area. Of 
course, there can be no large holes it there are no holes 
a1 all, so the euler runber can be used to see if a test 
for significant holes Is needed, U nf ortunateJ y, a laroe 
percentage of regions generated by real Images will hava 
small holes In them, especially near the edge, so this 
test will not raject very many. If a region Is known to 
hai/e no major holes, and has a high eccentricity, then It 
must be elongated* In the absence of a "major hole 11 
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predicate, there Is always the possibility that a high 
eccen + r ic Ft v ffl 3V be due to a perfectly round region, but 
with a large hole In the middle* 

In genera I , any sort of s hape-de ser i ptd r 
algorithm can be applied to the regions, such as the Glum 
algorithm {Medial An Ts Transform) [J3* I believe^ 
however, that one of the strengths of the Iraage tree a? a 
method is to allow par. y re-cogn It f on with relatively 
simple region shape d esc r (:p tor s-< Using very complicated 
descriptors not only will consume, a great dear of 
computer time, but will also complicate the analysis 
required of the higher- 1 eve I programs. A more detailed 
shape analysis should probably he reserved 1 for cases in 
which problems arise in the simpler procedures* 

5,4.2 Reg lon-ho I e Dua I Ity 

The tree procedures are not symmetric with 
respect to I Ight and dark, as has been pointed out 
earlier^ Thus 3 blacK spot on a I Tght object is not 
perceived as en object, but as a hole Fn a region* 
furthermore, these holes are rot detected by the 
programs, and Insufficient Information is stored on the 
tree to tell that they are there. Thus the effect of a 
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hole Is to decrease the ronton area t and increase the 
region eccentricity* but ft Is not delected as a hole per 
se* In the detection of texture., black speckles have a 
completely d Ifferent effect than white speckles* An 
c eject is harder to recognize on n white background thpn 
on a dark one. 

This. Is Tiot a deslreahle situation, An object 
should be easy to recognize on any highly contrasting 
bac kgrou nd, regardless of whether It Is darker or lighter 
than the object, A possible solution would be to make 
two trees, one with the rmege negated. Thus one wou Id be 
the tree already discussed In detsH, and the other would 
be a tree of dark regions on lighter backgrounds, in 
which the tips of the branches would represent I oca My 
dark areai, rather then locally light ones* For the face 
considered |n section 4+4 t these dark branches would 
represent significant locally dark areas, such as the eye 
sockets, the nostrils, and the dark areas along the side 
of the nose* The eye sockets and the nostrils, In 
particular, are prob-ably very Important In orienting 
visually with respect td ej fa:e. 

There is re reason why this procedure shojld not 
be carried to more than one level. Whenever a region Is 
Isolated, the contiguity scan routines could be called 
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again, but scanning only inside the region, and with 
their sense inverted, so that they would find holes. 
Small ho les cou Id then b eel Imitated, b-ut if therevere 
any I e<* go ones, they wOu Id be noted on the trea* 
Furthermore, the sense COUld ther be inverted cnca mora, 
and the contiguity scan triad once again to find 
additional I f 5 h 1 regions Inside the dork holes, 

This procedure would Succeed tn finding a dark 
apple on e light background. The apple could be Isolated 
by an Inverted run of the tree procedures, and then the 
normal procedure eoti id be carried out on the region thus 
I so I fifed . 

5.4.3 Complex Lighting 

In the ahove discussion. It was assumed 1 that the 
Illumination *as coming from g slnqle point source. 
Changing the source of the. F I I urn I na t Ion will change the 
properties of the highlight region, but will not a H er 
the baste properties of the tree. If the Illumination 's 
from a diffuse source, specularity Inforir Et icn is lost. 
Light frcfl several point Sources w1 II produce multiple 
highlights. If the high level parsing rou tines know 
about the light source, they cen compensate for these 
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effects. By mfl k Prig hypotheses about the objects in the 
image, these routines Could equally well 1 I nd out about 
the lighting from the imago, 

*i.4.L Isolations of Regions 



A by-product of the Image tree Js the Isolation 
of reg Ions *htch can be used as data foe other feature 
extraction programs. One mighty for Chippie, take a 
fairly large region around the hichl font, subtract Out 
the small region containing the hlghl Ighi Itself^ and 
hand this difference region to a textural analysis 
program. This program could use this region to extract 
texture Information In various hrtySp such as performing a 
Fourier transform, autocoi^vo I ut 1 on , or similar 
processing,, obtaining Information about surface speckles 
not available directly from the tree, Using- a region 
generated from one of the t<-ee roces helps assure that 
■the portion of the Tmepe upon which the analysis fs 
perfcrned Is a suitable one. 
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5*5 Summary a fid Conclusions 



> .. . ...,_. cr -| ^ r - ,., -r.r, teen c-'l'r.r-j *u? p -' oc [> S f r n 

ime-ges of three-dimensional objects >H th snpothly curved 
surfaces* The rmoThod Ts able to extract soma Information 
about the surface properties of the objects, such ns the 
tesctjre, specular rty, and surface Irregu lar rty . 
Information about shape fs a l &o attracted* The 
procedures are insensitive to noise eno d I stort lorij and 
can be used To perform real recognition tasks. It Is 
hoped that this work will provide a stepping-stone In the 
challenging study of c om puter vision. 
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Append ik : Description of Algorithms 



This appendix contains an outline of the 
algorithms used in the tree generating program. 

The Image tree Is generated one threshold level 
gt a time, starting et the highest level (branch tips). 
At each level, the image ts scanned, and the poFnts above 
the threshold are marked In a scratch array. This 
scratch array Ts than scanned for marked poFnts, When one 
Is found j a eontlguity rgytFne Fs called, which visits 
all marked points which can be reached f r cm the start via 
a connected path. The narks are erased 1 by this routine 
as It goes, and statistics are kept on the region thus 
generated, such as the sums of the x end y coordinates of 
the points, end the sum pf the souares of the x end y 
coordFnates (used to compute the center of mass and the 
eccentric Ity)* A tree node is then made up for the 
region, and the scan for marked poinds continues, A 
special mark Is left In the scratch array for each 
region, When this mark Ts encountered during the scan 
at the nej<t I ev e 1 1 It 1s looked up on an association 
Mst 4 This establishes the link between a region an<? 
the regions which are a subset of [t at the previous 
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l^vel - l*e, between- a node and Its sub-ncdes* 

The contiguity scan Is the most complex program. 
It works by leaving directional pointers In the scratch 
array* These are three-bit codBs demoting one of the 
eight possible neighboring points. The COfltlgylty Stan 
Is always started at a point which fa cm the bottom edge 
of the region. It traces along this edge to the right by 
moving frcn one marked polftt to the next, but always 
keeping an un-marked point to the right side, As It 
goes, ft erases the marks, so that for a region with 
smooth boundaries, it kIM fol low a spiral path to The 
center, "eatTng up 11 the marks as It goes. Ilka a lathe 
with the tool continually advancing Into the work* 

As the contigu Ity routlnE scans, It lays dOurn 
bock pointers in the scratch array which enable It to 
retrace Its path back to the start. If a dead end Is 
res chad Cno mora marked neighbors). It traces back alone 
this pathj looking for narked points to the right. There 
can be no marked points on the left side while 
bae ktrac kFng, since this was the Mnht side on the way 
out, and the outgoing scan stayed as far to the right as 
possible. If a marked point ts found on the backi-rBiCe, 
It Is replaced with a pointer to the adjacent path 
already traced out, and than a new path Is traced as If 
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this wer* a ncv starting point* When the, backtrace 
reaches the original starting point, the contFguity scan 
Is completed. The effect of this algorithm Is to 
construct a tre& 'F pointers In the scratch ^rray, with 
the starting point at the- root. All points which can be 
reached via a connected path from the starting point will 
be a P=rt of thl e tree, an example of which Is shown In 
f Igur e A I * 

An algorithm developed! by £ „ Bryan [ 4 J coultf 
speed the contiguity scan considerably, l+ entails 
Coding the scratch array Mne by line as strips, as In 
figure A2 . Each strip Is specif led by Its y coordinate, 
and the x coordinates of Its left and right end. The 
contiguity of these strips is then checked, rather then 
operating on the Individual points. This algorithm not 
only avoids scanning the entire scratch array, most of 
which fs blank, but also reeulres fewer operations to 
find all of the contiguous points, since they or* 
gathered into groups. It thus takes advantage of the 
fact that reclons produced by real Images, as opposed to 
random nolsa, will tend to have the points clustered! Into 
faunehe s. 

A number of other programs were written In the 
course of this research* In er<Jer to make it convenient 
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Figure AL; The Tree of Pointers Layed Down by the Contiguity 

Scan Algorithm! 



(Shown for an arbitrary region) 



* = marked poi^t^ included in region 

\ = pointer In direction of root 

(arrowhead not shawn due to small si*e) 
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FtguTC- A2 : A Region lidded as Strips 



The same region is used as in £igute A.1 
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+0 Study a largo numbor of trees* programs were written 
*c print Out the trees on the \ i ne-pr 1 n ter , uFth the 
significant parameters associated with each nods* 
Fu rthermore, & program was. produced to p 1 et any parameter 
vs. threshold along any set of branches of the tree. 
This program was used to produce the graphs In this 
paper . 

Programs were aTso written to display art 
Intensity module + ed picture of the Image, using the Seven 
Intensity levels of a DEC 34D display* Since our 340 has 
no fast raster mode, a display compiler was written which 
generates 5 display list In Increment mode, allowing 
fairly large Images to be shown virtually f I ic ker- f r ee f 
Other routines enable any arbitrary region ir the Image 
to be shown Superimposed on this picture. The pointer 
method used I n the contiguity Scan was actually written 
for thas* display routines, which were developed first b 
The existence of this program made the wrlt|r»c] of the 
Contiguity scan very Simple, which Is one reason why 
faster algorithms such as the Bryan algorithm ye re not 
soug ht* 

A large amount of code was required to back up 
the programs mentioned above. This Includes a dynamic 
storage a! locator for manipulating a large number cf 
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arrays of Changing *fze, display a nd plotter rcgtrnes. and 
ofner I/O routines, routines for manipulating list 
structure,, and routines which map arbitrary lota ! 
procedures over an array. The programs comprise over 
520 words Of PDP-IQ M IDA5 assembly lannuf>™ code, not 
including about P700 words of fixed buffer and tables, 
end not including the dynamically al!oca"Ed array and 
list structure sroa, which can grow to an arbitrary stse* 

Also used yas + he CNTOUR program EI2H, whTch 
draws Intensfty contour rreps of an image, end Which was 
written early in the course cf this research, before the 
exact arfia of Study had been decked upor, 
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